System and Method for A High Dynamic Range Sensitive Sensor Element Array

ABSTRACT

A high dynamic range sensitive sensor array is described which uses a combination of pixel structures, some of which incorporating phase domain integration techniques to accurately capture high and low intensity images. The sensor elements included as many of the pixel structures in the sensor array are not limited by dynamic range characteristics exhibited by prior art solid-state pixel structures and is thus capable of capturing a full spectrum of electromagnetic radiation to provide a high quality output image.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation in part of, and claims priority under35 U.S.C. §1.20 to, U.S. patent application Ser. No. 11/533866 filedSep. 21, 2006 which claims priority under 35 U.S.C. §1.119(e) toprovisional patent application Nos. 60/719,306, 60/719,304 and60/719,305 filed on Sep. 21, 2005 by Davidovici et al, and toprovisional patent application Ser. No. 60/727,897 filed Oct. 18, 2005by Davidovici et al. All of the above identified priority applicationsare incorporated herein by reference.

FIELD OF THE INVENTION

This invention relates generally to the field of electronic imaging andmore particularly to a method and apparatus for enhanced image captureusing photometric measurement and reporting.

BACKGROUND OF THE INVENTION

Photography is the process of making pictures by means of the action oflight.

Light is the commonly used term for electromagnetic radiation in afrequency range that is visible to the human eye. Light patternsreflected or emitted from objects are recorded by an image sensorthrough a timed exposure. Image sensors can be chemical in nature, suchas photographic film, or solid state in nature, such as the CCD and CMOSimage sensors employed by digital still and video cameras.

Digital cameras have a series of lenses that focus light to create animage of a scene. But instead of focusing this light onto a piece offilm, as in traditional cameras, it focuses it onto the image sensorwhich converts the electromagnetic radiation of the light into anelectrical charge. The image sensor is said to be a picture element, ora ‘pixel.’ The electrical charge indicates a relative intensity of theelectromagnetic radiation as perceived by the image sensor, andgenerally is used to associate a light intensity value with the pixel.

FIG. 1 illustrates typical component blocks that may be included in adigital image processing system 10. The system 10 includes a signalsource 100 and a signal processing chain that consists of integrator110, analog to digital converter (ADC) 120 and DSP 130. Signal source100 could for example be a sensor such as a light intensity sensor thatgenerates an electrical response in response to electromagneticradiation, such as light, impinging upon it. For the purposes of thisapplication, the system 10 shall be referred to as a conventional pixelstructure.

The output of integrator 110, V_(OUT), is input to ADC 120. ADC 120performs the analog to digital conversion function. The analog todigital conversion function is well known in the art. The analog signalV_(OUT) present at ADC 120 input is converted into signal V_(D) that cantake one of a set of discrete levels.

The quality of the signal is improved by integrator 110 which integratesthe signal V_(IN). FIG. 2 illustrates the nature of the signalimprovement. Waveform 200 is the combination of a constant value signalgenerated by signal source 100 and additive noise that corrupts theconstant value signal. Waveform 210 is the integrator output generatedin response to input signal waveform 200. It is readily observed thatsignal fluctuations caused by the additive noise decrease in waveform210.

Signal source 100 could be a light intensity sensor that is used in atimed application, such as in a digital camera application where thesensor is exposed to the light for a specific duration of time, commonlyreferred to as the exposure time. The integrator 110 then also servesthe function of integrating the response of sensor 100 caused by allphotons received during the exposure time into one value, such as forexample a voltage, to be read-out at the end of the exposure time.

FIGS. 3 illustrates a typical image sensor circuit. Signal source 1000is a light sensor that by way of example can be said to be a photodiode.Capacitor 1040 is a simple integrator. The input to the integrator isthe output of signal source 1000. Capacitor 1040 is reset by switch 1050which is in the closed position prior to starting the integrationprocess. At the start of the integration process switch 1050 opens andthe voltage across capacitor 1040 begins to change in response to theinput signal originating from signal source 1000. At the end of theintegration process switch 1030 closes and integrator output 1060,V_(OUT), is sampled. FIG. 3 is an illustrative diagram. Theimplementation of other similar integrators with identical functionalityis well known to one skilled in the art.

Integrator output 1060, V_(OUT), cannot in general exceed the upperlimit imposed by the available power supply voltage. Power supplyvoltages are decreasing in state-of-the-art equipment due to stringentpower consumption requirements. Integrator output 1060 cannot exceed thepower supply voltage and will saturate if the integrator output signalcontinues to build after reaching the power supply voltage level. Thesaturation condition is illustrated in FIG. 4A. Saturation occurs whenthe output voltage reaches the available power supply voltage and isunable to increase any further in response to the input signal. Signalsaturation causes system performance degradation. FIGS. 4A through 4Cillustrate potential distortions at the output of a pixel structureconsisting of light sensor 100 and integrator 110 due to the dynamicrange limitation of the photosensitive element structure and morespecifically of the integrator structure.

Segment (a) of FIG. 4A illustrates the linear increase of integrator 110output in response to a constant input signal of different level. Theimage sensor structure will perform well for the range of input lightintensities that give rise to the linear output of segment (a); theimage sensor structure will not perform well for the range of inputlight intensities that give rise to the saturated output of segment (b).

The integrator output response is indicative of limited dynamic range.As illustrated in FIG. 4A the image sensor will render well shadowdetail but will fail to render highlight detail. It is possible to shiftthe response as illustrated FIGS. 4B and 4C. In FIGS. 4B and 4C thedynamic range of the image sensor remains the same but the responsecharacteristic is shifted. The response characteristic of FIG. 4B losesshadow and highlight detail but retains good midrange response. Theresponse characteristic of FIG. 4C loses shadow detail and partialmidrange detail in order to maintain good highlight detail.

FIG. 5A illustrates the histogram of the pixel intensities of anoverexposed image capture where a multitude of pixels were driven intosaturation, such as in FIG. 4A. As seen in FIG. 5A the maximum pixelstructure output value is ‘255’ and the units used are the ADC 120output codes corresponding to the pixel output voltage. The lightintensity caused many light sensors 100 to output a value that saturatedthe integrator 110 as the exposure progressed during the exposureperiod. The maximum (saturated) value of the integrator 110 outputcaused the ADC to generate the output code ‘255’ which is the maximumoutput code for an 8-bit ADC. The image capture will be of suboptimalquality due to the inability of those pixels subject to high intensitylight inputs to achieve a sufficiently high output level. A lowerintegrator 110 gain would have caused the outputs of the light imagesensor subject to high intensity light inputs to register a below-255output and avoid the high end distortion.

FIG. 5B illustrates the histogram of the pixel intensities of anunderexposed image capture where a multitude of pixels were not exposedto sufficient light to achieve a minimum output value. As seen in FIG.5B the minimum pixel structure output value is ‘0’ and the units usedare the ADC 120 output codes corresponding to the pixel output voltage.The light intensity caused many light sensors 100 to output a value thatfailed to cause integrator 110 to output a sufficiently high value tocause a minimal ADC output code as the exposure progressed during theexposure period.

The image capture will be of suboptimal quality due to the inability ofthose pixels subject to low intensity light inputs to achieve asufficiently high output level. The distortion illustrated in thehistogram of FIG. 5B corresponds to the individual pixel distortion ofFIG. 4C. A higher integrator 110 gain would have caused the outputs ofthe light image sensor, subject to low intensity light inputs, toregister an above-zero output and avoid the low end distortion.

FIG. 6 illustrates the response curve of a pixel structure built usingdouble-slope technology. The nonlinear extension of dynamic rangeillustrated in FIG. 6 avoids saturation effects; however, the non-linearrelationship between the intensity of the electromagnetic energyimpinging upon the sensor and the sensor's output causes the image to becaptured with reduced resolution when high levels of light intensity arepresent.

Other approaches such as multiple exposure combining, conditional slopeswitching and logarithmic response pixel structures have been published.The multiple exposure combining, conditional slope switching andlogarithmic response pixel structures exhibit performance degradationsthat render them unsuitable for high performance image acquisitiontasks.

Integrator saturation is the limiting factor in the dynamic rangeperformance of a pixel structure. Solutions to the integrator saturationproblem have been published. One feature the published solutions have incommon is the monitoring of the integrator output to detect the onset ofsaturation condition at which time the integrator is discharged and theevent is recorded. This class of solutions is difficult to implementefficiently in integrated circuits (ICs) due to accuracy requirements ofanalog components and non-standard analog implementations. Theimplementation of accurate comparators that operate in a noisyenvironment near the power supply voltage, where integrator outputsbegin to saturate, is a difficult undertaking that consumes excessivepower, an undesirable operational feature.

Analog IC designs are difficult and time consuming to implement. It isadvantageous to use standard building blocks that have been fullydebugged and optimized for size, power consumption and performance. Theclass of published solutions does not meet this requirement.

SUMMARY OF THE INVENTION

According to one aspect of the invention, an image sensor array includesa plurality of pixel structures, at least one pixel structure includinga photosensitive element for generating a signal in response toelectromagnetic radiation and a a phase integrator, coupled to thephotosensitive element, for integrating the response of thephotosensitive element to the electromagnetic radiation over an exposuretime period using phase information. At least one pixel structure of theimage sensor array is adapted to avoid saturation when the signals fromthe photosensitive element are large.

According to another aspect of the invention, a method of capturing animage includes the steps of exposing an mage sensor array comprising aplurality of pixel structures to electromagnetic radiation to generatean image sensor array signal for an exposure period, includingprocessing signals obtained from the plurality of pixel structuresduring the exposure time period to generate an image, wherein theplurality of pixel structures includes at least one pixel structure thatis adapted to avoid saturation when the signals from the photosensitiveelement are large.

With such an arrangement, a solid-state image sensor array is providedthat is capable of producing a faithful reproduction of an image to becaptured regardless of the intensity of electromagnetic energy impingingon the sensor.

These and other advantages of the invention will be described withregard to the below figures.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a block diagram illustrating exemplary components that may beincluded in an image acquisition chain;

FIG. 2 is a graph illustrating the output of an integrator;

FIG. 3 is a block diagram of a typical pixel structure;

FIGS. 4A-4C are transfer curves provided to describe distortions at theoutput of a pixel structure such as that of FIG. 3, illustrating theeffect of dynamic range limitations;

FIGS. 5A and 5B are histograms of pixel intensities of respectiveoverexposed and underexposed image capture;

FIG. 6 is a response curve of a pixel structure built using double-slopetechnology;

FIG. 7A is a response curve of a pixel structure of the presentinvention;

FIG. 7B illustrates an exemplary histogram of pixel intensities in acaptured image of the present invention;

FIG. 8 is a block diagram of a pixel structure of the present inventionfollowed by ADC and DSP blocks;

FIG. 9 is a block diagram of one embodiment of a phase domain integratorof the present invention;

FIGS. 10A and 10B are graphs illustrating an oscillator phase (9A) and aintegrator output signal (9B), and is used to describe the signalintegration capabilities of the present invention;

FIG. 11A is a graph which illustrates an adjustable oscillator waveformthat varies in frequency in response to an oscillator input illustratedin FIG. 11B;

FIG. 12 is a graph illustrating the affects of quantization error on lowintensity signals;

FIGS. 13A-13D illustrate several common VCO output waveforms;

FIGS. 14A and 14B are graphs provided to illustrate a phase unwrappingmechanism of the present invention;

FIG. 15 is an exemplary block diagram of a sensor element of the presentinvention including a phase domain integrator;

FIG. 16 is a flow diagram provided to illustrate several steps that maybe performed during an image capture process by the phase domainintegrator of FIG. 14; and

FIG. 17 is a diagram of a mixed pixel image sensor array whichincorporates various forms of the pixel structures of the presentinvention.

DETAILED DESCRIPTION

According to one aspect of the invention, a high dynamic range capablesensor element or array is provided which uses phase domain integrationtechniques to accurately capture high and low intensity images. Thesensor element of the present invention is not limited by dynamic rangecharacteristics exhibited by prior art solid-state pixel structures andis thus capable of capturing a full range of electromagnetic radiationto provide a high quality output image.

FIG. 7A illustrates the response of a sensor element built using thetechnology of the present invention. The extended dynamic range of thesensor element is sufficient to enable it to respond to impingingradiation with a pixel response over the full range of electromagneticradiation intensity. As a result the sensor element is able to capturesufficient charges in the darkest portion while avoiding the saturationaffects in the brightness portions of the image to be captured. The neteffect it is faithful reproduction of the image to be capturedregardless of the relative intensity of the electromagnetic energyimpinging upon the sensor.

FIG. 7B illustrates the histogram of the pixel intensities of acorrectly exposed image capture where all pixel outputs are within thedynamic range of the 8-bit ADC that is zero to 255.

The sensor element of the present invention includes a novel integratorimplementation that is based on frequency oscillator circuits. Frequencyoscillator circuits are standard IC component blocks and do not sufferthe disadvantages of the prior art solid-state devices. Also, the novelsensor elements disclosed herein uses accurate integrators that canaccommodate output signal values far in excess of available power supplyvoltages and with very high accuracy and dynamic range.

For high performance image acquisition it is desirable to have imagesensors capable of high resolution and high dynamic range imageacquisition using a single sensor read-out step. FIG. 8 illustrates anexemplary pixel structure of the present invention. The pixel structureof FIG. 8 uses the conventional signal acquisition structure of FIG. 3but replaces the time domain integrator 110 with a novel phase domainintegrator 210. FIG. 9 shows the phase domain integrator of the presentinvention in more detail.

In the present invention, the output of signal source 100 (FIG. 8) isconnected to the input 800 of the phase domain integrator of FIG. 9. Atthe end of the integration period the integral of the input signal isread in phase format at the phase domain integrator output 840.

The image sensor assembly and specifically the integrator sectionsatisfy two criteria: a) produce a large output in response to weakinput signals from the light sensitive element and b) avoid saturationwhen the input signals from the light sensitive element are large. Thesetwo criteria are mutually exclusive in conventional solid-state imagesensor structures. However, the present invention realizes that the twocriteria can be satisfied through the use of a phase domain integratorsuch as that of FIG. 9.

The operation of the phase domain integrator can best be fullyunderstood by exploring the concepts of integration, phase and frequencyand their interrelation. The integration function, or integral, is amathematical function that is well known in the art.

Briefly, an integral is a mathematical object that can be interpreted asan area or a generalization of area. If a signal is plotted as a curve,the integral of the signal is the area under the curve. An integrator isa device that integrates a signal present at its input and produces anintegrated version of the input signal at its output.

Phase and frequency have a differential relationship. The total phasetraversed by an oscillator output V_(out) during a duration of time ΔTis mathematically given byΔθ=∫ f _(inst) dt=∫(f _(nom) +f _(gain) ·S _(in))dtwhere the integral limits are over the time duration ΔT.

Separating the integral termsΔθ=∫(f _(nom) +f _(gain) ·S _(in))dt=∫ f _(nom) dt+∫ f _(gain) ·S _(in)dt=K+f _(gain) ·∫ S _(in) dtwhere the term K is a constant that is a function of the constant valuef_(nom) and ΔT (the integration time) and is therefore well known.

For the special case where f_(nom)=0 then K=0 andΔθ=f _(gain) ·∫ S _(in) dt.

The second term consists of a constant value multiplier f_(gain) and theterm ∫ S_(in) dt which is the integral of the input signal S_(in). Theterm f_(gain)·∫ S_(in) dt can be easily obtained by subtracting thevalue of K from the Δθ value at the end of the time period ΔT:f _(gain) ·∫ S _(in) dt=Δθ−K and∫ S _(in) dt=(Δθ−K)/f _(gain)

For the special case f_(nom)=0 and therefore K=0∫ S _(in) dt=Δθ/f _(gain)

The relationship above establishes the differential relationship betweenthe VCO control input signal S_(in) and the phase Δθ traversed by theVCO or oscillator output during a period of time ΔT. FIGS. 10A and 10Billustrate this equivalence graphically. FIG. 10A plots the phasestraversed by the VCO output as a function of time. FIG. 10B plots theintegral with respect to time of the input control signal S_(IN).

FIG. 11A illustrates a voltage controlled oscillator output waveformgenerated in response to an input signal SIN illustrated in FIG. 11B.S_(In) is comprised of two constant value segments, the first segmentslabeled 720 in FIG. 11B being lower in value than the second segments,labeled 730 in FIG. 11B.

Referring back to FIG. 10B, integrator output segment 620 is theintegral output as a function of time of S_(IN) when the lower valueS_(IN) segment 720 was input to the integrator. Integrator outputsegment 630 is the integral output as a function of time of S_(IN) whenthe higher value S_(IN) segment 730 was input to the integrator.

The lower value S_(IN) segment 720 caused the VCO to oscillate at alower frequency than the higher value S_(IN) segment 730. Waveformsegment 700 in FIG. 11A illustrates the lower VCO oscillation frequency.The higher value S_(IN) segment 730 at the VCO input causes the VCO tooscillate at a higher frequency than the lower value S_(IN) segment.Waveform segment 710 in FIG. 11A illustrates the higher VCO oscillationfrequency.

FIG. 10A plots the phases traversed by the VCO as a function of time.

Segment 600 corresponds to VCO output segment 700. Segment 610corresponds to VCO output segment 710. Segment 600 indicates a lowerphase accumulation rate than segment 610. The phase accumulation rate isthe phase traversed by the VCO as a function of time and can beexpressed in units of radians per second. This is identical to thevelocity with which the VCO traverses a unit of phase is the frequencyof oscillation that is expressed in units of radians per second.

The VCO control signal input waveform segment 720 causes the VCO tooutput waveform segment 700. The plot of the VCO output phase as afunction of time generates the curve segment 600.

The VCO control signal input waveform segment 730 causes the VCO tooutput waveform segment 710. The plot of the VCO output phase as afunction of time generates the curve segment 610.

The waveforms plotted in FIGS. 10 a and 10 b are identical in shape andare related by the constant f_(gain) when f_(nom)=0 and therefore K=0;when f_(nom)≠0 and therefore K≠0 the waveforms plotted in FIGS. 10A and10B are related by the constants f_(gain) and K.

Accordingly it is realized that the time domain integral of the inputsignal S_(IN) is therefore functionally equivalent to the phase domainintegral of the input signal S_(IN). The time domain integral of theinput signal S_(IN) and the phase domain integral of the input signalS_(IN) are related through two constants, one of which equals zero forthe special case f_(nom)=0.

The use of the method disclosed herein to perform signal integration hasadvantages over conventional integrators and resolves difficultperformance issues associated with conventional integrators. Oneadvantage is the resolution of the potential to saturate the integratoroutput. VCO or oscillator outputs are strictly bound by upper and lowerlimits (peak values) which are not exceeded under any circumstances.Therefore output saturation conditions cannot occur.

Another advantage is the resolution of the issue of quantization noise.As illustrated in FIG. 12 low level signal 310 could suffer fromsignificant and unacceptable quantization noise, with little distinctionbetween output voltages caused by small variations in input intensities.Phase measurement based integration measures the phase traversed by theoscillator output Δθ during integration time ΔT. The phase traversed bythe oscillator output Δθ during integration time ΔT is proportional tothe integral of the input control signal during integration time ΔT andthe two are proportional. The minimum Δθ value occurs for the smallestintegral output. Butf _(gain) ·∫ S _(in) dt=Δθ−K

where K is a constant. Therefore the term Δθ−K can be independently setto a specific value for any given value of ∫ S_(in) dt, including itsminimum, by simply adjusting the VCO gain f_(gain). The ability to setthe gain of the integrator and hence the minimum value of the measuredintegrator output variable eliminates the quantization noise issueassociated with conventional integrators. It should be noted that in anygiven pixel array, a variety of pixels of the present invention may beprovided, each having a different adjusted gain, to control the outputresponse of the pixel to weak input signals.

Yet another advantage is that variable oscillator circuits are commonand fundamental building blocks of a wide variety of systems. Thereforethey are widely available and have been highly optimized.

Thus VCO-based integrators are far superior to conventional integratorsin quantization noise and dynamic range or lack of output saturation.Other advantages exist and are apparent to one versed in the art.

Oscillators are a class of circuits well known in the art. The output ofoscillator circuits can have a variety of shapes but they are allperiodic, meaning that the output waveform is repetitive. One repetitionof the output waveform comprises one oscillation cycle and the durationof a cycle is defined as its period of oscillation.

The frequency of oscillation, f_(osc), is defined as the number ofperiods of oscillation per unit time and it is usually measured in Hertz(periods of oscillation per second). By convention the angular frequencyof an oscillator is defined as ω=2πf_(osc) and one complete cycle ofoscillation traverses a phase angle θ of 2π radians.

Associated with an oscillator are initial conditions, that is the stateof the system at some arbitrary time, t=0. An example of an initialcondition might be the initial phase of the oscillator at t=0 measuredin radians.

FIGS. 13A, B, C and D illustrate common output waveforms of oscillatorcircuits. As well known in the art FIGS. 13 A, B, C and D illustrate theoutput waveform of sinusoidal, triangular, sawtooth and square waveoscillators. In all cases the peak voltage range shown is one volt.

The frequency of oscillation of electronic oscillator circuits can befixed or variable. A common oscillator with variable frequency ofoscillation is the voltage-controlled oscillator (VCO). At a minimum aVCO has a voltage input at which a signal voltage S_(in) controls thefrequency of oscillation. Voltage relates to current through Ohm's lawand a signal S_(in) can be said to control the frequency of oscillationthrough its current rather than voltage characteristic.

A VCO may also have a nominal frequency of oscillation f_(nom). The VCOoscillates at f_(nom) when the frequency control input S_(in) level isnot present or of a value that does not modify the frequency ofoscillation, such as for example zero volts. The nominal frequency ofoscillation can have any specified value including zero Hertz.

The output frequency of the VCO changes in response to amplitudevariations of the input signal. Thus the instantaneous frequency ofoscillation of the VCO will differ from the nominal frequency ofoscillation of the VCO by some value f_(delta) and will be given byf _(inst) =f _(nom) +f _(delta)

wheref _(delta) =f _(gain) ·S _(in).

In this example the term f_(delta) is measured in radians per second,f_(gain) is measured in radians per second per volt and S_(in) ismeasured in volts.

As described above, FIG. 11A illustrates the input and output signals ofa VCO. The frequency of oscillation of the VCO output changes inresponse to variations in the input signal amplitude. The VCO outputsegment 700 corresponds to input signal segment 720. The VCO outputsegment 710 corresponds to input signal segment 730.

The frequency of oscillation of VCO output segment 700 is lower than thefrequency of oscillation of VCO output segment 710. The amplitude ofinput signal segment 720 is lower than the amplitude of input signalsegment 730. Therefore f_(gain) has a positive value and the VCOfrequency of oscillation is directly proportional to the input controlsignal amplitude.

A VCO may also have additional inputs, such as RESET/ENABLE. Thefunction of RESET/ENABLE when in the RESET state is to reset the VCOoutput waveform to a predetermined voltage that can be any value withinthe peak-to-peak voltage range. The function of RESET/ENABLE when in theENABLE state is to enable the VCO output to oscillate.

A VCO has an output V_(out). During each period of oscillation V_(out)traverses an angular phase of 2π radians. This implies that the outputphase is measurable modulo 2π and oscillator output values at phasesthat are separated by exactly 2π are identical. FIGS. 13A-13Dillustrates several common VCO output waveforms. During a completeperiod each waveform in FIG. 13 traverses exactly 2π radians andwaveforms values at phases that are separated by exactly 2π areidentical for all waveforms.

The phase traversed by the oscillator output during a subset of oneperiod is determined by sampling the oscillator output at the twoinstances of time marking the beginning and end of the subset of oneperiod, identifying the phase associated with each sample andsubtracting the two phases.

The phase traversed by the oscillator output during a duration of timethat spans more than one period of oscillation can only be determinedmodulo 2π radians when using a method based on direct observation of theVCO output at two time instances. Thus an additional function is usedthat counts the number of periods or significant fractions of a periodtraversed by the VCO output in order to resolve the ambiguity.

A circuit that counts the number of periods or significant fractions ofa period traversed by the VCO output in a time interval, or ‘unwraps’the phase, can be readily implemented. Waveforms associated with such acircuit are illustrated in FIGS. 14A and 14B. The VCO output in FIG. 14Ais the triangular waveform. At times t=0, 0.5 T_(P) and T_(P) the VCOoutput waveform reaches states labeled 900, 910 and 920 corresponding toa travel of 0, π and 2π radians.

The output of the phase unwrapping circuit changes states at times t=0,0.5 T_(P) and T_(P) to levels of 0, V and 2V amplitude. The transitiontimes are labeled 930, 940 and 950 and they correspond to the VCO outputstates labeled 900, 910 and 920, respectively.

The relationship between the VCO output and the output of the phaseunwrapping circuit, illustrated over one period of the VCO output, canbe extended over any number of VCO output periods with the output of thephase unwrapping circuit increasing in value by a predetermined amounteach time the VCO output goes through its 0 and π (modulo 2π) phasevalues. It is known to one skilled in the art that there are alternativeways to implement the phase unwrapping function and mark the value ofthe unwrapped phase traversed by the VCO output.

The total phase traversed by the VCO output is given by the summation oftwo terms. The first term is the total unwrapped phase recorded by thephase unwrapping circuit. The second term is the total phase traversedby the VCO output since the last update of the phase unwrapping circuitoutput. This quantity can be unambiguously obtained by directmeasurement of the VCO output.

FIG. 15 incorporates a simplified block diagram of the VCO subset ofICL8038 a commercially available IC. Additional phase unwrapping, totalphase traversed and VCO RESET/ENABLE functions are added.

Current sources 860 and 855 charge and discharge, respectively,capacitor 845.

The charging and discharging of capacitor 845 is determined by switch865 which is controlled by flip-flop 825 and which connects currentsource 860 or 855 to capacitor 845.

Flip-flop 825 changes states when triggered by comparators 815 and 820.Comparator 815 is triggered when capacitor 845 reaches a predeterminedhigh voltage. Comparator 820 is triggered when capacitor 845 reaches apredetermined low voltage.

When comparator 815 is triggered flip-flop 825 changes state such as tocause switch 865 to close. Current 12 of current source 860 causescapacitor 845 to discharge thus causing the voltage across capacitor 845to decrease. The decrease of the voltage across capacitor 845immediately causes comparator 815 to change state.

When the voltage across capacitor 845 decreases to a sufficiently lowvalue comparator 820 is triggered. When comparator 820 is triggeredflip-flop 825 changes state such as to cause switch 865 to open. CurrentI₁ of current source 855 causes capacitor 845 to charge thus causing thevoltage across capacitor 845 to increase. The increase of the voltageacross capacitor 845 immediately causes comparator 820 to change state.

When the voltage across capacitor 845 increases to a sufficiently highvalue comparator 815 is again triggered causing flip-flop 825 to changestate and capacitor 845 charge/discharge cycle to repeat.

The relationship between the charge held by capacitor 845 and thevoltage across capacitor 845 is Q=C·V where C is the capacitance ofcapacitor 845 measured in Farads, Q is the charge held by capacitor 845measured in Coulombs and V is the voltage across capacitor 845 measuredin Volts.

The change in charge held by capacitor 845 due to a constant current Ithat flows for an interval of time ΔT is given by ΔQ=I·ΔT where ΔQ isthe change in charge held by capacitor 845 in Coulombs, I is the valueof the current in Amperes and ΔT is the interval of time of current flowin seconds. A constant current causes a linear change in the charge heldby capacitor 845 as a function of time. The linear change in the chargeheld by capacitor 845 as a function of time causes a linear change involtage across capacitor 845 as a function of time.

The constant value of currents I₁ and I₂ generated by current sources855 and 866 cause the voltage across capacitor 845 to increase anddecrease linearly generating a triangular waveform. If the net effectsof currents I₁ and I₂ are equal the rising and falling segments of thetriangular voltage waveform across capacitor 845 are symmetric asillustrated in FIG. 6 c. If the net effects of currents I₁ and I₂ arenot equal the rising and falling segments of the triangular voltagewaveform across capacitor 845 are asymmetric. In the limit as the neteffects of current I₁<< the net effects of current I₂ the triangularvoltage waveform across capacitor 845 tends to the sawtooth waveformillustrated in FIG. 6 b.

The time to charge and discharge capacitor 845 is determined by themagnitude of currents I₁ and I₂ generated by current sources 860 and855. The sum of the times required to charge and discharge capacitor 845to voltage levels that trigger comparators 815 and 820 determine theperiod of oscillation of the VCO. Therefore the magnitudes of currentsI₁ and I₂ determine the period and frequency of oscillation of the VCO.

The control signal applied at input 870 controls current sources 860 and855 and therefore controls the VCO frequency of oscillation. Althoughnot shown a simple voltage or current splitter as well known to oneversed in the art can be added between the control signal applied at 870and current sources 860 and 855 to adjust the waveform symmetry.

Reversal of the voltage across capacitor 845 is controlled by the stateof flip-flop 825. Counter 835 is triggered and modifies its output statecorrespondingly each time flip-flop 825 changes state. The change incounter 835 output state can be a modified voltage level as shown inFIG. 8 b. Other voltage level modification schemes can be used as longas separate states are resolvable. Counter 835 output can also be of adigital format consisting of a digital word containing B bits. In suchcase a change in its output state can be a binary number where differentstates differ in one or more bits.

The output state of counter 835 changes each time flip-flop 825 changesstates and therefore counts how many times the VCO output reached itsminimum and maximum values. If the output state of counter 835 is aneven number the output of the VCO traversed an integer multiple of 2πradians. The number of 2π radians traversed by the VCO output is thengiven by dividing the output count of counter 835 by two.

If the output state of counter 835 is an odd number the number of 2πradians traversed by the output of the VCO has an integer and afractional part. The integer part of the number of 2π radians traversedby the output of the VCO is given by dividing by two a number obtainedby subtracting one from the output state of counter 835. The fractionalpart of the number of 2π radians traversed by the output of the VCOdepends on the degree of asymmetry between the rising and fallingsegments of the waveform and can be readily obtained by one skilled inthe art. By way of example if the rising segment of the waveform takestwice as long as the falling segment of the waveform than then itrequires ⅔ of a period of oscillation to complete.

The voltage across capacitor 845 is proportional to the phase traversedby the VCO output following the last change in state of flip-flop 825.It can be readily obtained by one skilled in the art if a) the triggervoltages of comparators 815 and 820 (i.e., the maximum and minimumvoltages of the VCO output) and b) the asymmetry between the rising andfalling segments of the VCO output waveform are known. By way of exampleconsider that if a) the VCO output voltage is halfway between theminimum and maximum value on the rising segment of the waveform and b)the rising segment of the waveform takes twice as long as the fallingsegment of the waveform then the waveform measurement is at ⅓ of aperiod of oscillation.

The total phase traversed by the output of the VCO is obtained bysumming the phase traversed by the VCO output as recorded by the voltageto phase converter 840 and by counter and phase converter 835. Thisfunction is performed by summer 880 and made available at output 890.

Switch 850 resets capacitor 845 and therefore the VCO oscillator outputto an initial voltage output by voltage source 810. Flip-flop 825 isreset by signal 895. The initial voltage of voltage source 810 togetherwith the reset state of flip-flop 825 and the degree of asymmetrybetween the rising and falling segments of the VCO output waveform aresufficient to determine the initial phase of the VCO output waveform.This derivation is well known to one versed in the art.

As well known in the art and described in the ICL8038 application notesliterature sinusoidal, square and sawtooth waveforms are derived byadditional internal circuits using the basic triangular waveformdiscussed herein. Therefore the items addressed herein apply equally toother VCO output waveform shapes.

The present invention thus replaces the conventional integratorcomponent of a pixel structure with a high gain and high dynamic rangeintegrator that performs the time integration of the input signal in thephase domain.

FIG. 16 is a flow diagram which illustrates several exemplary steps thatmay be performed during a capture process 150 by a pixel structure thatuses the phase domain integrator of the present invention.

At the beginning of said exposure time (step 151), the VCO output andthe counter and phase converter 835 are reset. At step 152 thephotosensitive element is exposed to light and changes its electricalcharacteristics causing the VCO output to change frequency. Thephotosensitive element can be any element such as a photodiode, aphotogate, a phototransistor or a photoresistor. The present inventionis also related to a solid-state imaging device, such as a CMOS or MOSimaging device having a geometric configuration of pixels, at least partof the pixels having the structure described above.

At step 153 counter and phase converter 835 records the unwrapped phasetraversed by the VCO output. When it is determined at step 154 that theexposure frame ends the output of the VCO is translated to radians byvoltage to phase converter 840 at step 155. The outputs of the counterand phase converter 835 and voltage to phase converter 840 are added bytotal phase adder 880. At step 156 the phase domain integration resultmay be translated to the time domain integration result if so desired.Step 156 is shown in dashed lines to indicate that it is not a necessarystep of the capture process.

Accordingly, a method and apparatus has been described for obtaining aread-out signal of a solid-state pixel structure (including CCD, CMOS-and MOS-based pixel structures). The pixel structure includes at least aphotosensitive element with an output node, means to integrate theoutput node signal in the phase domain and means to read the phasedomain integration value. With such an arrangement a solid-state pixelis provided that is capable of producing a faithful reproduction of animage to be captured regardless of the intensity of electromagneticenergy impinging on the sensor.

Referring now to FIG. 17, a pixel array 1000 is shown to include aplurality of pixels, such as pixels 1001, 1005 and 1007, and a functionblock 1002.

Voltage state information is forwarded from the pixels to Function Block1002. Function block 1002, in one embodiment, communicates with systemcomponents external to the pixel array 1000 via interface bus 1003. Thefunction block 1002 is advantageously dynamically programmable, althoughthis is not a requirement. In one embodiment, the function block may beprogrammed to detect exposure termination trigger conditions. Theexposure termination trigger conditions can be selected a priori to theimage capture process, or alternatively can be adaptively selected inresponse to incoming information from the image capture process. By wayof example potential preset exposure termination conditions include butare not limited to: an indication that the maximum exposure time thathas been set a priori has been reached, an indication that all pixelsreported that minimum output voltage has been reached and one pixelreported that its output voltage saturation value has been reached, anindication that a preset number of pixels reported that outputsaturation voltage has been reached, an indication that a preset numberof pixels reported that minimum output voltage has been reached, anindication that a number of pixels larger than a first preset thresholdreported that output saturation voltage has been reached and a number ofpixels larger than a second preset threshold reported that minimumoutput voltage has been reached, and an indication that a combination ofexposure termination conditions has occurred.

According to one aspect of the invention, it is not a requirement thateach of the pixels in the array 1000 be identical; rather, it isanticipated that some of the pixels, for example pixel 1001 may beconventional pixels, some pixels, such as pixel 1005, may be pixelshaving gains adjusted to provide a large output in response to arelatively small input signal, and other pixels such as pixel 1007 maybe built using a voltage controlled oscillator to avoid saturation whenthe signals from the photosensitive element are large.

Having described various embodiments of the invention, it will beappreciate that although certain components and process steps have beendescribed the descriptions are representative only; other functionaldelineations or additional steps and components can be added by one ofskill in the art, and thus the present invention should not be limitedto the specific embodiments disclosed. The various representationalelements may be implemented in hardware, software running on a computer,or a combination thereof and modification to and variation of theillustrated embodiments may be made without departing from the inventiveconcepts herein disclosed. Accordingly, the invention should not beviewed as limited except by the scope and spirit of the appended claims.

1. An image sensor array comprising: a plurality of pixel structures, atleast one pixel structure including: a photosensitive element forgenerating a signal in response to electromagnetic radiation; and aphase integrator, coupled to the photosensitive element, for integratingthe response of the photosensitive element to the electromagneticradiation over an exposure time period using phase information; andwherein at least one pixel structure is adapted to avoid saturation whenthe signals from the photosensitive element are large.
 2. The imagesensor array of claim 1, wherein the image sensor array is a mixed imagesensor array and wherein at least one pixel structure is adapted toproduce a large output in response to weak input signals from thephotosensitive element.
 3. The mixed image sensor of claim 2, wherein atleast one pixel structure is a conventional pixel.
 4. A method ofcapturing an image includes the steps of: exposing an mage sensor arraycomprising a plurality of pixel structures to electromagnetic radiationto generate an image sensor array signal for an exposure period,including processing signals obtained from the plurality of pixelstructures during the exposure time period to generate an image, whereinthe plurality of pixel structures includes at least one pixel structurethat is adapted to avoid saturation when the signals from thephotosensitive element are large.
 5. The method of claim 4, wherein theimage sensor array is a mixed image sensor array and wherein at leastone pixel structure is adapted to produce a large output in response toweak input signals from the photosensitive element.
 6. The method ofclaim 5 wherein at least one pixel structure in the mixed image sensorarray is a conventional pixel.